The agreement "in principle" to begin a US troop withdrawal only needed President Donald Trump's approval, Khalilzad had announced on Monday.
The person who told The Associated Press of President Ashraf Ghani's postponed Washington trip was not authorised to talk to reporters and spoke on condition of anonymity.
Since Khalilzad's announcement on Monday, two horrific Taliban car bombings in the Afghan capital, Kabul - one of which killed a US service member - and objections to the deal from the Afghan government and several former US ambassadors to Afghanistan have put pressure on Khalilzad as many wonder whether a deal will truly bring peace.